Crime Data Analysis
Stuart Barnum
1/28/2018
The dataset for crime in Seattle in 2015 contains more than 2000 instances with missing coordinate data (latitude and longitude). As an aid in investigating whether there is a systematic relationship between missingness of this data and other variables in the dataset, I constructed various interactive plots, based on a data frame of tallies constructed with dplyr.
library(tidyverse)## ── Attaching packages ────────────────────────────── tidyverse 1.2.1 ──## ✔ ggplot2 2.2.1.9000 ✔ purrr 0.2.4
## ✔ tibble 1.4.2 ✔ dplyr 0.7.4
## ✔ tidyr 0.7.2 ✔ stringr 1.2.0
## ✔ readr 1.1.1 ✔ forcats 0.2.0## ── Conflicts ───────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()library(plotly)##
## Attaching package: 'plotly'## The following object is masked from 'package:ggplot2':
##
## last_plot## The following object is masked from 'package:stats':
##
## filter## The following object is masked from 'package:graphics':
##
## layoutsanfrancisco <- read.csv("sanfrancisco_incidents_summer_2014.csv")
seattle <- read.csv("seattle_incidents_summer_2014.csv")
#To avoid repetition of code in the generation various conditional distributions, I
#defined a function that produces both the dataframes with the tallies and the interactive plots for
#examining the possible relations between missingness and other factors.
calculate_conditional_distribution <- function(column) {
column <- enquo(column)
zero_assess <- seattle %>%
mutate(Position_Data_Missing = Latitude == 0) %>%
group_by(Position_Data_Missing, !!column) %>%
summarize(m = n()) %>%
group_by(!!column) %>%
mutate(n = sum(m)) %>%
ungroup() %>%
mutate(sum_total = sum(m)) %>%
group_by(Position_Data_Missing) %>%
mutate(sum_zero_test = sum(m)) %>%
mutate(total_dist = n / sum_total) %>%
mutate(conditional_distribution = m / sum_zero_test) %>%
ungroup() %>%
group_by(!!column)
#initialize list of items to be returned (a d.f. and a ggplot)
return_list <- list()
return_list[["df"]] <- zero_assess
#create visualizations for assessment of independence
plot <- zero_assess %>%
filter(n > 10) %>%
mutate(Position_Data_Missing =
ifelse(Position_Data_Missing == TRUE,
"missing", "not missing")) %>%
ggplot() +
geom_col(aes_(x = column, y = ~conditional_distribution,
fill = ~Position_Data_Missing)) +
labs(x = str_c(quo_name(column),
"\n(move cursor over bars to see the categories)"),
y = "conditional distribution") +
ggtitle("Assessing the Effect of Missingness of Position Data") +
theme(plot.title = element_text(hjust = 0.5),
plot.subtitle = element_text(hjust = 0.5),
axis.text.x = element_blank(),
legend.title = element_blank()) +
guides(fill = guide_legend(reverse=TRUE))
return_list[["ggplot"]] <- plot
interactive_plot <-
ggplotly(plot) %>%
layout(margin = list(b = 50, l = 60, r = 10, t = 80))
return_list[["plotly"]] <- interactive_plot
return(return_list)
}We may thus find the distributions of certain general categories of crime, specfied by a “summarized offense description”, conditional on both missingness of and non-missingness.
results <- calculate_conditional_distribution(Summarized.Offense.Description)
results[["plotly"]]Overserving that the heights of the red and green portions are roughly the same in most of the cases, we find little evidence of a systematic relationship between missingness of coordinate data and the particular type of reported crime. However, moving the cursor over the bars, we see that prostitution is a notable exception, with perhaps all of the cases recorded with coordinate data. Examination of the data frame generated with the above code shows that there were 202 cases of prostition during the year (from 12/29/2014 through 1/2/2015), and indeed that all of the recorded cases had coordinate data.
Are there other types of crime for which this is the case? Let’s see.
#A simple manipulation of the data derived in calculate_conditional_distribution suffices.
results <- calculate_conditional_distribution(Summarized.Offense.Description)
library(knitr)
kable(results[["df"]] %>% filter(m == n))| Position_Data_Missing | Summarized.Offense.Description | m | n | sum_total | sum_zero_test | total_dist | conditional_distribution |
|---|---|---|---|---|---|---|---|
| FALSE | [INC - CASE DC USE ONLY] | 5 | 5 | 32779 | 30729 | 0.0001525 | 0.0001627 |
| FALSE | DISORDERLY CONDUCT | 2 | 2 | 32779 | 30729 | 0.0000610 | 0.0000651 |
| FALSE | DUI | 34 | 34 | 32779 | 30729 | 0.0010372 | 0.0011064 |
| FALSE | ELUDING | 8 | 8 | 32779 | 30729 | 0.0002441 | 0.0002603 |
| FALSE | ESCAPE | 3 | 3 | 32779 | 30729 | 0.0000915 | 0.0000976 |
| FALSE | HOMICIDE | 8 | 8 | 32779 | 30729 | 0.0002441 | 0.0002603 |
| FALSE | PORNOGRAPHY | 3 | 3 | 32779 | 30729 | 0.0000915 | 0.0000976 |
| FALSE | PROSTITUTION | 202 | 202 | 32779 | 30729 | 0.0061625 | 0.0065736 |
| FALSE | PUBLIC NUISANCE | 4 | 4 | 32779 | 30729 | 0.0001220 | 0.0001302 |
Although a chi-squared test may be useful here, what we have so far suggests at least a weak systematic relationship between missingness and the particular crime reported in certain instances. Perhaps prostitution was recorded with somewhat greater meticulousness? Although we can’t infer that this is the case, the data contains a suggestion that it might be.
We now turn to the relationship betwen district sector and missingness.
results <- calculate_conditional_distribution(District.Sector)
results[["plotly"]]The mostly-red bar at the left end corresonds to data for which no district sector is indicated in the data, for which it would not be surprising that the coordinate data is also missing. Other than, maybe, the mysterious District Sector 99–with a total of only 40 reported instances–the data suggests that the reporting of examples was consistent accross the sectors.
Similar considerations apply to the possible relationship between missingness and Zone Beat.
results <- calculate_conditional_distribution(Zone.Beat)
results[["plotly"]]In this case, minor differences in the conditional distributions are evident, but likely not significant. (I may conduct a Chi Squared test of this assumption.) We may thus reasonably suppose that a plot of the crime data on a map of the city would not distort the distribution of actual recorded crime across Seattle. Thus plotting all of the instantances that have position data (latitude and longitude) in the table, we obtain the following.
library(ggmap)##
## Attaching package: 'ggmap'## The following object is masked from 'package:plotly':
##
## wind#obtain a suitable Seattle map
seattle_gg <- get_map("Seattle", maptype = "toner-lite",
zoom = 11)## maptype = "toner-lite" is only available with source = "stamen".## resetting to source = "stamen"...## Map from URL : http://maps.googleapis.com/maps/api/staticmap?center=Seattle&zoom=11&size=640x640&scale=2&maptype=terrain&sensor=false## Information from URL : http://maps.googleapis.com/maps/api/geocode/json?address=Seattle&sensor=false## Map from URL : http://tile.stamen.com/toner-lite/11/326/714.png## Map from URL : http://tile.stamen.com/toner-lite/11/327/714.png## Map from URL : http://tile.stamen.com/toner-lite/11/328/714.png## Map from URL : http://tile.stamen.com/toner-lite/11/329/714.png## Map from URL : http://tile.stamen.com/toner-lite/11/326/715.png## Map from URL : http://tile.stamen.com/toner-lite/11/327/715.png## Map from URL : http://tile.stamen.com/toner-lite/11/328/715.png## Map from URL : http://tile.stamen.com/toner-lite/11/329/715.png## Map from URL : http://tile.stamen.com/toner-lite/11/326/716.png## Map from URL : http://tile.stamen.com/toner-lite/11/327/716.png## Map from URL : http://tile.stamen.com/toner-lite/11/328/716.png## Map from URL : http://tile.stamen.com/toner-lite/11/329/716.png#remove zero latitude and longitude elements, for mapping
seattle_nonzero <- seattle %>%
filter(Latitude != 0 & Longitude != 0)
#show the distribution of recorded reports of crime on the map, using a logarithmic scale to enhance
#the contrast
plot <- ggmap(seattle_gg, darken = c(.01, "black")) +
geom_bin2d(data = seattle_nonzero,
aes(x = Longitude, y = Latitude),
bins = 200) +
scale_fill_gradient(trans = "log10")
plot
Now focuss on violent crimes, with these defined as crimes with “Summarized Offense Description” of Assault, Homicide, or Robbery.
library(ggmap)
Violent_Crimes <- c("ASSAULT", "HOMICIDE", "ROBBERY")
data <- seattle_nonzero %>%
filter(is.element(Summarized.Offense.Description, Violent_Crimes))
ggmap(seattle_gg, darken = c(.01, "black")) +
geom_bin2d(data = data,
aes(x = Longitude, y = Latitude),
bins = 200) +
scale_fill_gradient(trans = "log10")
And drugs:
data <- seattle_nonzero %>%
filter(Summarized.Offense.Description == "NARCOTICS")
ggmap(seattle_gg, darken = c(.01, "black")) +
geom_bin2d(data = data,
aes(x = Longitude, y = Latitude),
bins = 200) +
scale_fill_gradient(trans = "log10")